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Response  times  to  the  items  in  a  series  of  computer  administered  question¬ 
naires  were  measured  in  two  continuous  sustained  operation  studies.  These 
studies  involved  subjects  performing  repeated  exercise  at  30%  of  maximum 
aerobic  power  as  well  as  subjects  doing  no  exercise.  Two  start  times  (noon  and 
midnight)  were  also  used.  Thirty  subjects  were  tested  for  33  one  hour  contin¬ 
uous  work  sessions  over  two  days  with  a  three  hour  nap  allowed  between  the 
days.  The  questionnaire  scales  selected  for  analysis  included  the  Vigor  and 
Fatigue  scales  of  the  Profile  of  Mood  States  (POMS),  the  Kogi  Symptom  Check¬ 
list,  the  School  of  Aerospace  Medicine  (SAM)  Fatigue  scale,  and  the  Naval 
Health  Research  Center  Mood  Questionnaire  (NHRC  MQ)  Negative  and  Positive  Mood 
scales.  These  scales  were  selected  to  reflect  the  major  experimental  conditions 
(sleep  loss  and  exercise  fatigue)  and  they  included  the  three  different 
response  scaling  types  (Likert,  Dichotomous,  and  Guttman). 

Response  times  to  the  scales  were  not  significantly  related  to  reading 
test  speed  or  grade  level.  The  response  times  to  almost  all  scales  were 
significantly  faster  on  the  second  workday.  The  symptom,  negative  mood,  and 
fatigue  scale  values  increased,  and  the  Vigor  and  positive  scale  scores 
decreased  between  the  first  and  second  days.  Reaction  times  on  a  simple 
psychomotor  task  were  significantly  slower  during  the  second  workday  and  did 
not  correlate  with  the  questionnaire  response  times.  This  finding  suggested 
that  response  times  to  questionnaire  items  are  not  based  on  the  same  mechanism 
as  psychomotor  reaction  times. 

Response  times  for  those  subjects  using  straight  lining  wore  significantly 
faster  than  for  those  using  variable  response  patterns.  Although  the  incidence 
of  this  straight-lining  increased  significantly  over  the  two  workdays,  it  did 
not  occur  often  enough  to  explain  the  shorter  response  times  during  the  second 
workday.  The  decrease  remained  significant  even  after  those  who  were  straight 
lining  were  omitted  from  the  analyses. 

There  were  significant  positive  correlations  between  scale  values  and 
scale  response  times  on  the  first  day  for  the  POMS  Fatigue,  Kogi  Symptom,  and 
NHRC  Negative  scales;  subjects  with  longer  response  times  had  higher  scale 
scores.  In  contrast,  the  NHRC  Positive  scale  value  and  the  response  time  to 
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that  scale  were  negatively  related  the  first  day.  Thus,  scale  content  influ¬ 
ences  the  direction  of  the  relationship  of  scale  value  and  response  time. 
Positive  mood  response  times  appear  to  reflect  the  intensity  of  feeling  (the 
higher  the  scale  value,  the  faster  the  response),  whereas  negative  affect 
responses  may  be  moderated  by  denial  or  deliberation  in  response  level 
selection.  Only  the  POMS  Fatigue  scale  showed  significant  (p<.001)  value- 
response  time  correlation  the  second  day.  This  scale  was  particularly  valid 
because  it  closely  reflected  the  experimental  conditions  (fatigue  and  sleep 
loss).  Thus  the  scale  value-response  time  relationship  may  be  a  measure  of 
scale  validity  in  particular  experimental  situations. 

The  Guttman  scaled  items  had  the  longest  response  times,  and  the  Kogi 
Symptom  dichotomous  scaled  items  the  shortest  response  times,  indicating  that 
the  more  complex  the  scale  type,  the  longer  the  response  times.  When  the 
average  response  times  were  compared  between  scales  having  the  same  format  but 
different  contents,  the  NHRC  Positive  scale  took  longer  to  respond  to,  per 
item,  than  the  Negative  scale,  and  the  POMS  Vigor  took  longer  than  POMS  Fatigue 
scale  both  days.  This  supports  Bower's  theory  (1981)  that  subjects  respond 
more  quickly  to  questions  that  reflect  moods  congruent  with  their  present  state 
(i.e.,  increased  fatigue  in  these  studies). 

Response  times  to  questionnaire  scales  were  shown  to  be  important  measures 
in  indicating  intensity,  denial  and  congruence  with  the  scale  contents,  as  well 
as  showing  the  influence  of  straight-lining  and  reflecting  the  experimental 
conditions  and  scale  difficulty  and  validity.  Response  times  were  not  found  to 
be  due  to  reaction  time,  reading  speed,  or  ability. 
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COMPUTER  RESPONSE  TIME  HEASURE!^ENT  OF  MOOD,  FATIGUE  AND  SYMPTOM 
SCALE  ITEMS:  IMPLICATIONS  FOR  SCALE  RESPONSE  TIME  USES 


David  H.  Ryman,  Paul  Naitoh,  Carl  E.  Englund  and  S.  G.  Genser 

Reaction  times  have  lore;  been  used  by  psychologists  to  measure  vigilance/ 
attention,  cognitive  processing,  and  skill  levels.  Shorter  reaction  times  have 
been  found  to  indicate  greater  vigilance  and  attention  as  well  as  higher  skill 
levels.  More  complex  cognitive  processing  has  been  found  to  cause  longer 
reaction  times. 

Although  computer  administered  questionnaires  hove  been  in  use  for  over 
twenty  years  and  computer  administered  reaction  time  measures  have  been  avail¬ 
able  equally  long,  there  is  scant  literature  on  the  measurement  or  use  of 
response  times^  to  questionnaire  scales.  Depressives  have  been  shown  to  have 
longer  response  latencies  to  psychomotor  reaction  time  measures  (Ban,  1964), 
but  the  relationships  between  response  times  and  questionnaire  scale  values 
with  depression  or  other  psychological  states  or  behaviors  are  not  well 
defined. 

Dower  (19R1)  proposed  the  theory  that  emotional  states  would  influence 
activation  of  memory  for  words  congruent  with  the  mood  state  and  inhibit 
opposing  mood  state  words.  In  such  a  theory,  response  times  to  mood  scales 
items  reflecting  the  prevailing  moods  would  be  expected  to  be  faster,  and 
response  time  to  opposite  moods  slower.  Response  time  to  mood  scale  items 
might  therefore  reflect  the  intensity,  denial  or  other  dimensions  of  affective 
states.  For  example,  extremely  fast  response  time  for  the  items  of  a  particu¬ 
lar  mood  may  indicate  that  a  subject  felt  that  mood  strongly;  it  clearly 
applied  to  his  present  feelings  ->r  that  tne  subject  was  sensitive  to  the  mood 
and  denied  it. 


Footnote:  1.  In  this  report,  the  term  "response  time"  refers  to  the  time 
between  the  presentation  of  a  questionnaire  item  on  a  CRT  and  the  key  press 
r'^'flecting  tne  questionnaire  response  scale  by  a  subject.  The  term  "reaction 
is  used  for  the  psychomotor  t.:.sk  time  between  stimuli  sre-sentat  ion  on  the 
screen  and  the  subjects'  keyboard  reaction  to  the  stimuli. 


However,  fast  response  times  may  also  occur  when  subjects  want  to  quickly 
complete  the  questionnaire.  This  is  called  "straight  lining."  Extremely  slow 
response  times  to  items  of  a  particular  mood  could  also  be  interpreted  in 
several  ways.  A  subject  may  have  had  a  difficult  time  deciding  on  a  response 
to  an  item  because  it  was  complex  or  had  unfamiliar  words,  or  the  subject  had 
difficulty  deciding  between  which  response  scale  level  to  choose. 

Gilbert  (196?)  had  previously  reported  findings  which  supported  Bower's 
theory  using  a  mechanical  presentation  and  timing  method  for  the  Taylor 
Manifest  Anxiety  Scale.  He  developed  "latency  weighted"  scoring  using  a 
combination  of  response  time  and  strength  of  agreement  or  disagreement  to  test 
items.  These  "latency  weighted"  scores  showed  correlations  from  .76  to  .86 
with  regular  non-weighted  scores  which  did  not  incorporate  response  times. 
However,  low  anxiety  subjects  had  significantly  higher  regular  scores,  whereas 
high  anxiety  subjects  had  higher  "latency  weighted"  scores. 

Dunn,  Lushene  and  O'Neil  (1972)  reported  on  relationships  of  MMPI  item 
response  latencies  and  item  characteristics  (length,  social  deviant  content) 
using  computer  presentation  and  timing  of  the  MI-1PI.  They  found  that  the  length 
of  the  items  (number  of  letters  and  words)  accounted  for  ^18-687  of  the  response 
time  variability,  and  the  social  desirability  of  the  items  accounted  for  only  a 
slight  but  significant  amount  (4-v'j).  They  did  not  report  the  relationships 
between  response  latencies  to  the  standard  scales,  but  "deviant"  responses  to  a 
subset  of  58  items  were  found  to  have  greater  response  times  trian  "non-deviant" 
responses. 

The  Specific  purposes  for  the  inclusion  of  scale  item  response  time 
measurement  in  present  research  were  to; 

1)  Determine  if  response  times  to  items  reflect  intensity 
(i.e.,  greater  scale  value)  in  acceptance  or  denial  of 
the  mood,  fatieu*  ,  or  symptom  state  expressed  by  trie 
item  content. 

2)  Investigate  whether  respons-’  times  are  related  to 
scale  conten*  inlicating  congruence  of  tne  scal(' 


3)  Determine  if  extremely  short  response  times  indicate 
hasty,  "straight-lining"  response  patterns. 

4)  Determine  if  scale  response  times  reflect  changes  in 
experimental  conditions  such  as  the  effects  of  sleep 
loss,  exercise-induced  fati^^ue,  circadian  phase,  and 
start-time  of  work  shift. 

5)  Determine  whether  response  times  to  questionnaire 
items  were  related  to  the  reaction  times  in  psycho¬ 
motor  tasks  or  with  reading  speed  or  grade  level. 

Response  times  and  the  scale  values  were  collected  in  two  studies  involv¬ 
ing  sleep  loss,  exercise,  and  shifts  in  workday  start  times.  The  computer 
administered  and  timed  questionnaires  used  in  these  studies  included  two  Likert 
scaled  mood  questionnaires,  a  dichotomously  scaled  symptom  checklist,  and  a 
Guttman  scaled  fatigue  questionnaire.  Reaction  times  on  a  simple  psychomotor 
task  were  also  collected  via  computer  immediately  after  the  questionnaire 
administration . 

METHOD 

Subjects 

There  were  15  pairs  (N=30)  of  informed,  volunteer  U.S.  Marine  Corps 
enlisted  personnel  who  completed  a  week  long  laboratory  study.  Their  mean  age 
was  21.2  yrs  (SD=1.2),  and  they  had  an  average  of  11.1  yrs  (SD=1.3)  of 
education  with  an  average  reading  grade  level  of  10.9  (SD=2.5)  on  the 

Nelson-Denny  Reading  Test  (Brown,  1966). 

Procedures 

One  subject  of  each  pair  was  randomly  chosen  as  the  exercise  subject. 
This  subject  exercised  by  walking  on  a  treadmill  carrying  a  22.7  kg  (50-lb) 
pack  for  the  first  1/2  hour  of  each  of  the  33  one  hour  task  sessions  over  the 
two  continuous  workdays.  This  exercise  subject  walked  at  a  rate  and  grade  that 
required  301  of  his  maximum  aerobic  capacity.  The  other  (control)  subject  did 
the  same  tasks  while  seated  at  a  computer  terminal  throughout  the  sessions. 
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The  first  study  had  seven  pairs  of  subjects  who  started  the  two  continuous 
workdays  of  the  study  at  midnight  (2400  hr),  whereas  the  second  study  involved 
eight  pairs  of  subjects  who  started  the  continuous  workdays  at  1300  hr. 
Subjects  in  both  studies  spent  the  first  day  performing  8  to  12  one  hour  ses¬ 
sions  to  familiarize  them  with  the  tasks  and  schedule.  There  were  eight  hours 
of  sleep  allowed  between  this  training  day  and  the  first  continuous  workday.  A 
3hr  nap  was  also  given  between  the  two  continuous  workdays. 

Questionnaire  Description 

A  series  of  mood,  symptom,  and  fatigue  questionnaires  were  administered 
and  timed  by  a  MING  11  computer  (Digital  Equipment  Corporation  Modular 
Instrumentation  Computer)  with  a  programmable  clock  (MNCKW)  and  two  CRT 
terminals  (for  more  details  on  the  task  battery  and  computer  system,  see  Ryman, 
Naitoh  and  Englund,  1984). 

The  first  questionnaire  presented  was  the  Profile  of  Mood  States  (POMS; 
McNair,  Lorr  and  Droppleman,  1971).  The  65  items  appeared,  one  at  a  time, 
after  the  instructions  on  use  of  the  5  point  Likert  type  scale.  The  subjects 
were  to  press  the  number  key  corresponding  to  how  each  adjective  described 
"your  present  state  now”,  then  hit  the  enter  or  return  key.  The  response  time 
to  each  item  was  the  time  from  the  moment  the  adjective  appeared  on  the  screen 
to  the  subject's  typing  of  the  number  key.  If  a  value  was  mistakenly  entered 
or  the  subject  changed  his  mind,  the  delete  key  could  be  hit  and  the  timing  was 
restarted  for  that  item.  Only  after  the  subject  pressed  the  return  key  was  the 
response  time  and  the  item  scale  value  stored  in  the  computer,  and  then  the 
next  item  presented. 

The  second  questionnaire  presented  was  the  10-itai)  School  of  Aerospace 
Medicine  Subjective  Fatigue  Checklist  (SAM  Fatigue;  Pearson  and  Byars,  1956) 
which  has  a  Guttman-type  scale.  This  questionnaire  was  presented  in  the  same 
manner  as  the  POMS.  The  next  questionnaire  was  the  Naval  Health  Research  Mood 
(Jjestionnaire  (NHRC  MQ;  Moses,  Lubin,  Naitoh  and  Johnson,  1974).  The  NHRC  MQ 
has  a  four  point  Likert  response  scale  with  the  same  wording  as  the  POMS  scale, 
except  there  is  no  mid-point  value.  The  last  questionnaire  presented  was  the 
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Kogi  Symptom  Checklist  (Kogi,  Saito,  and  Mitsuhashi,  1970)  which  is  a  set  of  30 
symptoms  found  to  increase  with  work  fatigue. 

If  the  subject  gave  an  invalid  (out  of  range)  response  to  an  item  on  any 
of  these  four  questionnaires,  the  response  was  rejected,  the  item  reappeared  on 
the  screen,  and  timing  wa'?  restarted.  Timing  was  done  with  a  programmable 
clock  board  with  the  time  to  respond  stored  in  milliseconds  in  the  higher  bits 
(4-16)  and  the  scale  value  response  in  the  lower  bits  (0-3). 

Scaling  Methods 

Guttman  scales  are  developed  by  administering  a  large  number  of  items 
reflecting  a  given  measure  to  a  population  of  raters.  These  raters  sort  and 
rate  items  as  to  what  level  of  a  measure  (fatigue  in  the  SAM  questionnaire) 
they  represent.  Items  on  which  most  raters  agreed  are  then  used  to  construct  a 
scale  range  of  approximately  equidistant  levels.  Subjects  are  given  a  dicho¬ 
tomous  or,  in  this  case,  trichotomous  (0  =  better  than,  1  =  same  as,  and  2  = 
worse  than)  response  choice  on  each  item. 

A  Lickert-type  scale  has  three  or  more  equally  appearing  response  alter¬ 
natives.  Lickert  scales  are  usually  developed  by  the  factor  analysis  of  the 
responses  to  a  pool  of  such  items,  grouping  the  highly  correlated  items  into 
scales.  The  NHRC  MQ  scales  were  generated  from  the  principal  components 
solution  (Moses,  Lubin,  Naitoh  and  Johnson,  1974)  whereas  the  POMS  scales  were 
derived  by  varimax  (orthogonal)  rotation  (McNair,  Lorr  and  Droppleman,  1971). 

Scoring  Methods 

Scoring  programs  computed  the  response  times  on  the  scales  and  the  total 
scale  scores  for  the  six  POMS  scales,  the  total  SAM  Fatigue  scale,  the  negative 
and  positive  scales  of  the  NHRC  MQ,  and  the  total  number  of  symptoms  on  the 
KOGI.  The  data  from  these  questionnaires  were  stored  on  floppy  disk  media. 
The  scale  scores  and  times  were  output  to  another  floppy  and  transferred  to  a 
larger  VAX  computer  system  where  further  analysis  was  done.  Since  the  first 
item  in  a  questionnaire  appeared  with  the  instructions,  the  response  time  to 
the  first  item  in  a  questionnaire  was  not  included  in  its  scale  total  response 


time.  This  was  done  because  the  reading  of  questionnaire  instructions  and 
adjustment  from  one  questionnaire  response  scale  to  the  next  would  affect  those 
scales  having  the  first  questionnaire  item. 

Psychomotor  Reaction  Time  Task 

The  Simple  Reaction  Time  task  (Lisper  and  Kjellberg,  1972),  modified  for 
visual  stimuli,  was  administered  by  computer  immediately  after  the  series  of 
Mood,  Fatigue  and  Svimptom  questionnaires  were  finished.  There  are  60  trials  at 
random  intervals  with  a  mean  between  trial  interval  of  6  seconds.  Subjects 
respond  as  quickly  as  possible  to  a  stimulus  consisting  of  the  presentation  on 
a  CRT  screen  of  clock  ticks  in  lOQths  of  a  second  increasing  from  000  to  250 
(2.5  secs)  or  until  the  suoject  presses  any  key  on  the  terminal  keyboard.  The 
subject's  reaction  time,  or  250  for  an  omission,  remaineo  on  the  CRT  screen  for 
.5  seconds  after  each  trial.  The  most  sleep  loss  sensitive  measure  for  this 
task  nas  been  found  to  be  the  mean  reaction  time  of  the  slowest  10*  of  the 
responses.  This  measure  was  related  to  the  scale  response  times  to  estimate 
the  psychomotor  reaction  time  component  of  scale  response  times. 

ANALYSIS 

Skewness  and  kurtosis  of  scale  value  and  response  time  distributions  were 
similar  and  almost  all  were  in  the  range  of  -1  to  +1.  Hence,  these  response 
times  were  not  log-transformed  before  statistical  comparisons  as  is  usually 
done  for  reaction  time  measures  prior  to  analysis. 

Effects  of  Experimental  Conditions 

Multivariate  analyses  of  variance  (MANOVA)  of  Continuous  Work  Start-Times 
by  exercise  group  over  the  repeated  day  and  session  periods  (Norusis,  1985) 
were  done  on  the  response  times  for  the  six  selected  scales  to  determine  if 
response  times  differed  between  the  experimental  conditions  or  changed  with 
accumulating  fatigue  and  sleep  loss. 
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In  order  to  produce  more  stable  measures,  data  for  each  subject  from  the 
two  continuous  work  days  were  condensed  into  the  means  of  eight  time  periods  as 
follows: 


Period  1 :  sessions 
Period  2:  sess:'ons 
Period  3:  sessions 
Period  4:  sessions 
Period  5:  sessions 
Period  6:  sessions 
Period  7:  sessions 
Period  8:  sessions 


1-4  of  continuous  work  day  1 . 
5-8  of  continuous  work  day  1 . 
9-12  of  continuous  work  day  1. 
13-17  of  continuous  work  day  1. 
1-4  of  continuous  work  day  2. 
5-8  of  continuous  work  day  2. 
9-12  of  continuous  work  day  2. 
13-16  of  continuous  work  day  2. 


Interrelationships  of  Scale  Values  and  Response  Times : 


Correlations  between  the  means  of  daily  scale  scores  and  the  means  of 
daily  response  times  were  calculated  for  each  scale.  Pearson  Product-Moment 
Correlation  Coefficients  (rp)  were  obtained  to  test  for  any  linear  relation¬ 
ships  between  response  times  and  scale  values.  The  Spearman  Rank  Order  Corre¬ 
lation  (r^)  was  also  calculated  to  confirm  Pearson  correlations  because  the 
Spearman  is  not  influenced  by  extreme  values  (high  or  low  outliers)  as  is  the 
Pearson  and,  therefore,  is  more  robust  for  small  samples.  The  interrelation¬ 
ships  of  scale  values  and  response  times  were  accepted  only  when  both 
correlation  coefficients  were  significant. 

Effects  of  Reaction  Time 

Correlations  of  the  daily  mean  response  times  for  the  scales  and  the  mean 
reaction  time  for  the  slowest  10%  trials  on  the  Simple  Reaction  Time  (SRT)  task 
were  computed.  This  determined  if  reaction  time  was  a  significant  component  of 
mood,  fatigue,  or  symptom  scale  item  response  times. 

Effects  of  Reading  Ability 

Correlations  with  the  Nelson-Denny  Reading  Test  number-of-items-attempted 
and  grade  level  score  were  correlated  with  the  daily  mean  response  times  to  the 


six  scales,  to  determine  if  reading  speed  or  ability  was  related  to  the  scale 
item  response  times. 

T-Tests  of  Extreme  Response  Style  on  Time  and  Scale  Values; 

The  use  of  only  one  response  alternative  to  respond  to  all  the  itans  for 
all  the  scales  of  the  questionnaire  was  defined  as  straight-lining  that  ques¬ 
tionnaire.  If  the  questionnaire  only  had  one  scale  (Kogi  and  SAM),  shifting 
from  a  variable  response  pattern  to  the  use  of  only  one  response  alternative 
was  also  considered  straight-lining.  The  frequency  of  straight-lining  was 
analyzed  at  each  session  for  each  questionnaire.  The  scale  values  and  response 
times  of  straight-liners  were  compared  by  t-test  with  all  other  scale  response 
times.  Other  analyses  which  showed  significant  results  were  verified  by 
repeating  the  analyses  with  straight-lining  subjects  excluded. 

Comparisons  of  Mean  Item  Response  Times  of  Scales: 

The  mean  item  response  times  for  each  scale  for  each  day  were  computed  by 
dividing  the  number  of  items  in  a  scale  into  the  mean  response  time  for  the 
scale.  This  allowed  comparisons  between  the  response  times  across  the  various 
scaling  types  (four  and  five  point  Likert,  Guttman,  Dichotomy)  and  between  the 
various  scale  contents. 

RESULTS 

Scale  Characteristics ,  Values  and  Response  Time  Differences 

Items  in  the  four  scales  using  the  Likert  response  scaling  method  were 
comparable  in  number  and  length  of  words.  The  Guttman  scaled  SAM  Fatigue  and 
di chotomously  scaled  KOGI  symptom  checklist  were  also  similar  in  this  regard 

(Table  1). 

Thie  Kogi  symptom,  NHRC  Negative  mood  and  SAM  and  POMS  fatigue  scales 
scores  showed  significant  increases  over  the  two  continuous  workdays,  whereas 
tne  POMS  Vigor  and  NHRC  Positive  scores  decreased.  The  reaction  times  to  all 
scales,  however,  showed  significant  decreases  between  these  days  despite  the 


TABLE  1 

MOOD,  FATIGUE  AND  SYMPTOM  SCALE  CHARACTERISTICS 


OUESTiONNAIRE 

POMS 

SAM 

KOGI 

NHRC  MO 

SCALE 

VIGOR 

FATIGUE 

FATIGUE 

SYMPTOMS 

NEGATIVE  POSITIVE 

NO.  OF  ITEMS 

9 

7 

10 

30 

10 

19 

MEAN  NO.  WORDS 

1.3 

1.1 

2.2 

2.7 

1.0 

1.6 

MEAN  NO.  LETTERS 

7.4 

7.3 

12.7 

16.3 

6.2 

9.5 

SCALE  RESPONSE  TYPE 

LIKERT  5* 

GUTTMAN" 

DICHOTOMY* 

LIKERT  4* 

*0  =  Not  at  all;  1  =  A  little;  2  =  Moderately;  3  =  Quite  a  bit;  and  4  =  Extremely 
“0  =  Better  than;  1  =  Same  as;  and  2  =  Worse  than 
'Y  =  Yes;  and  N  =  No 

“O  =  Not  at  all;  1  =  A  little;  2  =  Quite  a  bit;  and  3  =  Extremely 


sleep  loss  and  exercise  fatigue  experienced.  The  correlation  between  mean 
response  times  day  1  and  day  2  for  a  given  scale  were  very  similar  to  the 
correlation  between  the  scale  scores  on  the  two  days  (see  Table  2). 

Differences  in  Scale  Response  Time  Across  Experimental  Factors; 

The  most  significant  factor  seen  from  the  HANOVA  analysis  of  these  scale 
response  times  was  the  Day  effect.  Response  times  wore  faster  during  continu¬ 
ous  work  day  2  (CW2)  than  during  continuous  work  day  1  ((XI).  Table  2  shows 
means  and  standard  deviations  for  scale  scores  and  response  times. 


TABLE  2 

SCALE  VALUE  AND  RESPONSE  TIME*  MEANS  AND  STANDARD  DEVIATIONS 
FOR  THE  TWO  CONTINUOUS  WORKDAYS  WITH  T-TESTS  AND  CORRELATIONS 

BETWEEN  THE  DAYS 


QUESTIONNAIRE 


DAY  CW1  CW2 

MEASURE  MEAN  SO  MEAN  SO  t  p  r 


4.35  <-001  .635 


Fatigue 

Fatigue 

Value 

RT 

8.3 

2.4 

3.3 

.9 

10.3 

2.1 

3.9 

1.0 

-3.47 

2.93 

.002 

.007 

.647  < 

.828  < 

.001 

.001 

Symptoms 

Value 

2.4 

2.0 

6.3 

7.1 

—3.09 

.004 

.186 

Symptoms 

RT 

.9 

.5 

.6 

.4 

2.17 

.039 

.760  < 

.001 

Negative 

Value 

5.2 

4.9 

6.8 

5.0 

1.98 

- 

.596  < 

.001 

Negative 

RT 

1.4 

.5 

1.3 

.4 

2.73 

.011 

.572  < 

.001 

Positive 

Value 

33.9 

13.6 

25.7 

13.3 

5.19  <.001 

.791  < 

.001 

Positive 

RT 

1.7 

.06 

1.5 

.5 

3.35 

.002 

.800  < 

.001 

Slowest 

10%  rt* 

.69 

.43 

1.01 

.56 

-5.19  <001 

.806  < 

.001 

df  =  29 

df  =  28 

=  Response  Time  per  questionnaire  item  in  seconds 
=  Reaction  Time  to  SRT  psychomotor  task  in  seconds 
=  not  significant  (p  >  .05.  two-tailed) 

=  Simple  Reaction  Time  Task  (mean  of  the  slowest  10%  responses) 

The  effect  of  workday  start-time  was  seen  in  the  significant  interaction 
of  Day  with  Start  Time.  The  noon  start  time  group  showed  much  longer  response 
times  on  continuous  workday  1,  as  compared  to  workday  2  for  POMS  Fatigue  and 
Vigor  and  NHRC  Negative  and  Kogi  Symptctn  scales,  than  the  midnight  group  did 
(Table  3)*  This  indicated  that  the  Day  effect  previously  discussed  occurred 
primarily  in  the  noon  group  for  these  four  scales. 

Session  (see  Analysis  Section)  differences  were  seen  for  the  Kogi  and  both 
NHRC  scales  with  the  last  sessions  taking  longer.  Longer  response  times  to  the 
POf-IS  fatigue  for  the  noon  start  time  group  over  the  last  four  sessions  were 
also  shown  by  the  significant  Start  Time  by  Days  by  Sessions  interaction. 
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TABLE  3 

SELECTED  MANOVA  RESULTS  FOR  RESPONSE  TIMES  TO  SIZE  SCALES 


QUESTIONNAIRE 

POMS 

SAM 

KOGI 

NHRC  MO 

— 1 
_ 1 

SCALE 

VIGOR 

FATIGUE 

FA-^IGUE 

SYMPTOMS 

NEGATIVE 

POSITIVE 

FACTORS 

F 

p 

F  P 

F  P 

F 

P 

F 

P 

f 

P 

Days' 

25.62 

<001 

4.22  .05 

10.03  .004 

5.05 

.03 

11.43 

.002 

15.15 

.001 

Days  X  ST* 

6.01 

.021 

4.21  .05 

.64 

6.53 

.02 

8.67 

.007 

3.69 

- 

Sessions 

1.41 

— 

2.80 

.99 

4.96 

.008 

4.47 

.012 

3.47 

.032 

'Response  times  faster  CW2  than  CW1  (see  Table  2  tor  means,  SDs)  pf  =  i,26 
'  =  Start  time  of  CW  (Midnight  vs  Noon  groups):  df  =  1.  26 
—  =  not  significant  (p  >  .05) 

Sessions  df  =  1,  24 

Relationships  Between  Scale  Response  Times  To  Reading  Scores  and  Scale  Values ; 

There  was  a  significant  linear  relationship  between  the  response  times  and 
scale  values  each  day  for  the  POMS  Fatigue  scale  (see  Figure  1).  Even  when  the 
Dunn-Bonferoni  correction  for  multiple  testing  (Harris,  1975)  was  applied  to 
the  significance  levels  (.05/12  or  p>.00ii),  this  relationship  remained  signi¬ 
ficant.  for  both  the  Pearson  and  Spearman  correlations.  Figure  1  shows  that 
correlations  result  from  subjects  with  higher  fatigue  having  consistently 
slower  response  times,  and  those  denying  fatigue  (zero  fatigue  scale)  respond¬ 
ing  very  quickly. 

The  Kogi  sy-uptom  scale  scores  had  a  significant  positive  correlation  with 
the  response  times  only  during  CW1  (Pearson  rp  =  .149,  p  =  .006;  Spearman  r^  = 
.65,  p  <  .001).  The  NHRC  Negative  scale  response  times  also  showed  a  positive 
correlation  with  scale  values  only  during  CW1  (rp  =  .36,  p  =  .05;  rg  =  -.46,  p 
=  .005).  There  was  a  significant  negative  correlation  (see  Figure  2)  between 
the  ,‘iHRC  Positive  scale  response  times  and  scale  values  (rp  =  -.40,  p  =  .03;  rg 
=  -.53,  P  =  .001)  during  the  first  continuous  workday.  During  CW1  higher  NHRC 
Positive  scale  values  were  associated  with  faster  responses,  while  higher  scale 
values  on  the  NHRC  Negative  and  Kogi  Symptom  scale  RTs  were  associated  with 
slower  (longer)  response  times.  All  correlations  between  the  scales'  Response 
Times  and  Values  for  both  days  can  be  seen  in  Table  4. 


POlITVEMOOORTfi 


MHflC  rosmVE  SCALE  CWt 


3.5 

3.0< 


NHRC  POSITIVE  SCrUJS  CW2 


Figure  2.  Regression  Line  with  95?  Confidence  Intervals  and  Data  Points  for 
Correlations  Between  Mean  NHRC  Positive  Mood  Response  and  Scale  Values  at  CW1 
(upper  graph)  and  CW2  (lower  graph). 


TABLE  4 

CORRELATIONS  BETWEEN  THE  DAY  MEAN  SCALE  VALUES  AND  RESPONSE  TIMES 


QUESTIONNAIRE 

POMS 

SAM 

KOGI 

NHRCMQ 

SCALE 

VIGOR 

FATIGUE 

FATIGUE 

SYMPTOMS 

NEGATIVE 

POSITIVE 

DAY 

fp  fp 

r„ 

r. 

fp 

r. 

rp 

r. 

rp 

r. 

fp 

r. 

CW1 

-.23  -.28 

.66 

.82 

.30 

.42 

.49 

.65 

.36 

.46 

-.40 

-.53 

P 

—  — 

<.001 

<.001 

— 

.01 

.005 

<001 

.05 

.006 

.03 

.001 

CW2 

.08  .02 

.56 

.61 

-.09 

-.12 

-.21 

-.03 

-.15 

-.07 

.20 

.03 

P 

—  — 

<.001 

<.001 

— 

— 

— 

— 

— 

— 

— 

— 

To  =  Pearson 
r«  =  Spearman 

~  =  not  significant  (p  >  .05.  two-tailed) 
df  =  28 


Scale  Response  Time  Relationships  With  Psychomotor  Reaction  Times  and  Reading 

The  mean  of  the  slowest  10%  reaction  times  on  the  Simple  Reaction  Time 
task  for  CW2  was  negatively  correlated  with  the  POMS  Vigor  scale  response  time. 
Those  subjects  with  faster  Vigor  scale  responses  had  slower  SRT  reaction  times 

P=.02  and  rg=-.i45,  p=.01).  There  were  no  other  significant  rela¬ 
tionships  between  the  psychomotor  reaction  times  and  the  scale  response  times. 
None  of  the  scale  response  times  were  significantly  correlated  with  the  Nelson- 
Denny  reading  test  grade  level  (ability)  or  number  of  answers  attempted 
(speed). 

Comparisons  of  Constant  Scale  Value  Responders  Versus  Others ; 

Straight-lining  occurred  in  less  than  2t  of  sessions  on  the  first  continu¬ 
ous  workday.  However,  during  the  second  workday  12  to  16%  of  all  sessions 
showed  such  response  patterns  (Table  5).  Four  subjects  (13.3%)  were  found  to 
use  straight-lining  in  responding  to  most  scales  (five  subjects  for  the  Kogi) 
for  a  majority  of  the  sessions  on  continuous  workday  2. 
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TABLE  5 

CW2  SCALE  VALUE  AND  RESPONSE  TIME  MEANS  AND  STANDARD  DEVIATIONS 
AND  T-TESTS  BETWEEN  'STRAIGHT-LINING'  AND  'OTHERS' 


QUESTIONNAIRE 

SCALE 

MEASURE 

OTHERS 

STRAIGHT-LININO 

MEAN 

so 

MEAN 

SO 

1 

P 

POMS 

Vigor 

Scale 

10.5 

6.3 

6.3 

4.8 

1.28 

.21 

POMS 

Vigor 

RT 

12.9 

3.9 

4.6 

2.7 

4.05 

.001 

POMS 

Fatigue 

Scale 

5.9 

4.5 

2.9 

3.4 

1.23 

.23 

POMS 

Fatigue 

RT 

9.7 

3.0 

4.9 

2.8 

3.02 

.005 

SAM 

Fatigue 

Scale 

10.9 

3.6 

6.5 

4.3 

2.24 

.03 

SAM 

Fatigue 

RT 

19.8 

8.1 

10.6 

8.8 

2.08 

.05 

NHRC 

Negative 

Scale 

6.9 

5.2 

6.2 

3.3 

.23 

.82 

NHRC 

Negative 

RT 

13.0 

34 

6.9 

2.5 

3.45 

.002 

NHRC 

Positive 

Scale 

27.2 

13.6 

15.5 

4.1 

1.69 

.10 

NHRC 

Positive 

RT 

27.2 

8.5 

12.8 

5.3 

3.26 

.003 

II 

c 

n  =  4 

df  = 

28 

KOGI 

Symptoms 

Scale 

6.1 

7.4 

7.4 

5.7 

-.40 

.70 

KOGI 

Symptoms 

RT 

23.0 

11.4 

15.7 

6.9 

1.37 

.18 

II 

C 

n  =  5 

df  = 

26 

•Use  of  constant  response  alternative  throughout  all  questionnaire  scales  lor  questionnaires  having  more  than  one  scale,  or  the 
sudden  switching  to  one  response  alternative  for  all  items  for  questionnaires  having  one  scale. 


The  straight-liner's  response  times  were  significantly  faster  (p<.001) 
than  the  response  times  of  those  using  more  than  one  response  alternative. 
When  the  subjects  using  straight-lining  were  excluded  from  the  analyses,  the 
significant  results  did  not  change  (slower  response  times  on  continuous  workday 
2  and  the  significant  correlations  between  response  times  and  scale  values 
remained) . 

Comparisons  of  Response  Time  Between  Scale  Contents  and  Scaling  Types : 

The  Postive  NHRC  MQ  scale  took  significantly  longer  than  the  Negative 
scale  on  both  days  (CW1,  t=2.66,  p=.003;  CW2,  t=2.l8,  p=.037),  and  the  POMS 
Vigor  scale  took  longer  to  complete  than  the  POMS  Fatigue  scale  each  day  (CW1, 
t=4.72,  p<.001;  CW2,  t=2.l8,  p<.037).  The  complex  Guttman  scaled  SAM  fatigue 
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items  had  a  longer  response  times  than  the  Likert  scaled  POMS  Fatigue  items 
(CW1,  t=6.08,  p<.001;  CW2,  t=M.07,  p<.001).  There  was  no  significant  differ¬ 
ence  in  the  response  time  per  item  between  the  NHRC  Negative  and  POMS  Fatigue 
scales,  but  the  Five  Point  Likert  scale  POMS  Vigor  scale  took  longer  per  item 
than  the  NHRC  Positive  (CW1,  t=  5.58,  p<  .001;  CW2,  t=  2.08,  p=  .047).  The 
response  times  to  the  dichotomous  scaled  symptom  items  were,  as  expected, 
significantly  faster  (p<.001)  than  with  all  other  scale  types. 

DISCUSSION 

The  primary  purpose  of  this  paper  was  to  investigate  how  measurement  of 
response  time  items  could  be  useful  in  such  areas  as:  1)  the  investigations  of 
intensity,  denial,  or  congruence  of  mood  states,  2)  the  occurrence  of  straight 
lining,  3)  experimental  effects  like  sleep  loss  or  exercise,  and  4)  its 
relationships  with  reaction  time  and  reading  measures.  Although  the  present 
studies  tiad  small  sample  sizes,  the  number  of  questionnaires  were  limited,  and 
there  were  many  statistical  tests  run  on  this  one  set  of  studies,  nevertheless, 
the  results  from  these  analyses  suggest  many  research  applications  for  scale 
response  time  as  a  significant  and  useful  measure. 

It  appears  that  the  scale  content  moderated  the  value-response  time  corre¬ 
lations  on  day  1.  The  negative  correlation  between  the  NHRC  positive  mood 
scale  and  the  response  time  to  that  scale  may  have  indicated  greater  intensity, 
higher  scale  value  having  faster  responses  on  the  NHRC  Positive  mood;  whereas 
the  negative  correlations  of  the  POMS  Fatigue,  Kogi  symptom  and  NHRC  Negative 
scale-response  times  might  have  been  due  to  denial  or  deliberation.  Perhaps 
subjects  tended  to  respond  quickly  to  sensitive  items  that  they  wanted  to  deny 
(give  a  low  score)  or,  conversely,  there  may  have  been  increased  deliberation 
between  higher  scale  values  on  negative  items  that  they  were  willing  to  apply 
to  themselves.  The  results  of  previous  studies  also  indicate  that  scale 
content  may  influence  the  relationship  with  response  time.  Gerrig  and  Bower 
( 1982)  did  not  find  any  differences  in  speed  or  correct  selection  of  words  with 
experimental  manipulation  of  happiness  or  anger.  However,  Ladavas,  Nicoletti, 
Umilta  and  Hizzolatti  (1984)  in  the  study  of  visual  field  differences  on  a 
reaction  time  task  found  a  slowing  of  reaction  time  post  self-induced  sadness 


as  opposed  to  initial  testing  and  post  recall  of  every  day  events.  They  attri¬ 
buted  this  finding  to  "an  activation  of  the  right  hemisphere  during  sadness", 
(pg.  484). 

The  relationship  of  response  time  to  scale  value  on  the  POMS  Fatigue  scale 
was  particularly  strong  (p<.001)  on  both  workdays.  Fatigue  was  a  major  factor 
in  our  experimental  protocol.  This  particular  correlation  suggests  that 
response  time  may  be  a  useful  indicator  of  scale  validity.  This  was  the  one 
scale  response  time  that  did  not  significantly  decrease  between  the  days. 
Response  times  of  all  other  scales  were  faster  during  CW2  than  during  CW1.  The 
correlations,  however,  between  response  times  and  scores  for  these  other  scales 
were  not  significant  during  CW2.  Thus,  for  the  other  scales,  sleep  loss  and 
exercise  fatigue  appear  to  affect  the  relationships  between  scale  values  and 
scale  response  times  differently  than  the  POMS  Fatigue  scale.  Figure  1  shows 
that  fast  response  time  with  denial  (zero  scale  value)  of  fatigue  as  well  as 
slow  responses  due  to  deliberation  with  higher  fatigue  responses  contribute  to 
the  significant  relationship. 

Significant  slowing  of  psychomptor  reaction  time  in  continuous  workday  2 
as  compared  to  1  was  found  in  the  present  study,  as  in  a  similar  earlier  study 
(Englund,  Naitoh,  Ryman  and  Hodgdon,  1983).  This  was  an  expected  result  of 
sleep  loss  and  exercise  fatigue.  In  contrast,  faster  scale  response  times  for 
most  scales  were  seen  on  CW2  as  compared  with  CW1.  Apparently,  questionnaire 
response  times  are  affected  differently  by  sleep  loss  and  exercise  fatigue  than 
psychomotor  reaction  times.  Possible  explanations  include:  increased  famil¬ 
iarity  with  the  questionnaires  and  scales,  or  less  inhibition  of  the  subjects 
to  admitting  poor  mood,  greater  fatigue  and  greatly  increased  number  of  symp¬ 
toms  making  the  questionnaires  more  pertinent.  Response  times  to  the  scales 
used  in  present  studies  were  also  not  related  to  reading  speed  or  ability. 

The  differences  in  scale  structure  played  a  major  role  in  response  speed. 
The  Guttman-scaled  SAM  Fatigue  questionnaire  showed  longer  response  times  than 
the  five  point  Likert  POMS  Fatigue  scale.  This  finding  probably  reflects  the 
comparative  nature  of  Guttman  scaling.  The  SAM  Fatigue  items  contain  longer 
descriptive  phrases  rather  than  the  one  word  adjectives  found  in  the  other  mood 
questionnaires  (see  Table  1).  Length  alone,  however,  does  not  appear  to  cause 


longer  response  times.  The  dichotomously  scaled  Kogi  items  had  the  most  words 
and  letters  per  scale  item  but  were,  by  far,  the  quickest  to  complete.  Dunn, 
et  al.,  (1972)  found  that  response  time  among  the  dichotomously  scaled  MMPI 
items  varied  most  with  item  length.  The  present  findings  comparing  different 
response  scales  indicate  that  scale  length  and  complexity  contribute  to  longer 
response  times. 

In  this  and  a  previous  similar  study  (Englund,  et  al.,  1983)  symptoms  and 
the  fatigue  and  negative  scale  values  increased  progressively,  while  Vigor  and 
Positive  scales  showed  decreased  scores.  The  response  times  to  the  scales  with 
negative  content  (fatigue,  negative  mood  scales)  were  faster  than  those  with 
positive  content  (vigor,  positive  mood  scales).  The  POMS  Fatigue  items  were 
more  quickly  answered  than  POMS  Vigor  items  during  both  first  and  second 
workdays.  The  response  times  of  NHRC  Negative  items  were  also  shorter  than 
those  for  NHRC  positive  items.  This  provides  some  support  for  Bower's  theory 
(1981)  of  moods  and  memory.  According  to  his  theory,  faster  responses  should 
be  found  for  moods  congruent  with  the  subject’s  current  state.  From  this 
theory  the  increasing  negative  scores  seen  in  this  study  would  be  expected  to 
be  accompanied  by  decreasing  response  times.  This  is  congruent  with  the 
subjects*  increasing  fatigue  and  lowered  vigor  as  sleep  loss  and  exercise- 
induced  fatigue  accumulate. 

As  expected,  most  of  the  extremely  short  response  times  were  the  result  of 
subjects'  straight-lining  responses  (Table  5).  Straight-lining  responders 
showed  significantly  faster  response  times  than  those  with  other  response 
patterns.  Surprisingly,  however,  the  scale  value  of  the  straight-lining 
responses  was  not  consistently  different  from  other  response  patterns.  This 
suggests  that  the  loss  of  correlation  between  response  times  and  scale  scores 
during  the  second  workday  cannot  be  totally  explained  by  the  increased 
incidence  of  straight-lining  responses. 

The  noon  start  group  showed  slower  mood  and  fatigue  response  times  during 
the  first  workday  (a  significant  Day  x  Start  Time  interaction)  as  compared  with 
those  starting  at  midnight.  This  could  be  due  to  a  circadian  rhythm  effect. 
The  noon  start  group  ended  the  first  continuous  workday  near  the  period  of  the 
circadian  trough  for  human  performance  (0300-0600),  whereas  the  midnight  group 


had  been  up  4  hours  before  they  entered  this  period  of  circadian  low.  The  noon 
group  had  significantly  faster  responses  to  these  scales  during  the  second  work 
day  than  they  had  during  the  first  workday.  For  the  midnight  group  this 
difference  was  not  significant.  This  suggests  that  sleep  loss  interacted  with 
the  circadian  effect  on  scale  response  time.  The  response  times  were  longer 
during  the  last  periods  of  each  day  for  both  Kogi  symptom  and  NHRC  scales. 
This  may  be  due  to  the  effect  of  being  awake  for  20  hours  or  a  loss  of 
concentration. 

SUMMARY 

These  results  demonstrate  how  response  times  to  questionnaire  scales 
indicate  reaction  to  scale  content  (congruence,  intensity  and  denial),  as  well 
as  showing  scale  difficulty  and  validity.  Questionnaire  response  times  were 
unlike  psychomotor  reaction  times  under  these  experimental  conditions  involving 
fatigue  and  sleep  loss.  These  results  indicate  that  response  times  to  ques¬ 
tionnaire  scales  could  be  a  useful  additional  measure  in  many  areas  of 
research.  With  the  increased  use  of  computers  in  the  administration  of 
questionnaires,  accurate  response  timing  can  easily  be  incorporated  into  a 
variety  of  research  protocols. 
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19.  ABSTRACT  (continued) 


with  scale  RTs.  The  significant  negative  correlation  between  the  POMS  Vigor  scale 
value  value  and  RT  scale  RT  CWl  indicated  some  support  to  RT  as  a  measure  of  inten¬ 
sity  of  mood  (faster  RT  with  higher  Vigor  scores).  The  NHRC  Positive  Scale  showed 
longer  RT  than  the  Negative  Scale  and  the  POMS  Vigor  took  longer  than  the  POMS 
Fatigue  scale  on  both  days,  consistent  with  the  theory  that  moods  congruent  with 
the  present  state  are  more  prominent  in  memory,  causing  faster  RT.  The  items  using 
the  most  complex  (Guttman)  scale  had  the  longest  RT  while  the  most  simply  (dicho¬ 
tomous)  scaled  items  had  the  shortest  RT.  The  results  demonstrate  the  RT  measure¬ 
ment  of  questionnaire  scales  can  be  used  as  a  measure  of  Intensity  and  congruence 
of  mood,  as  well  as  indicating  scale  validity.  RT  alone  was  not  useful  for  indi¬ 
cating  hasty  responders  (straight  liners),  but  subjects  using  a  constant  scale 
response  value  were  significantly  faster  (lower  RT) .  The  POMS  Fatigue  scale  and 
RT  were  signlficanly  correlated  on  both  days  indicating  the  usefulness  of  RT  in 
showing  scale  validity  (the  more  fatigue,  the  slower  the  response).  ^ 

Scale  item  RT  was  demonstrated  to  be  useful  in  showing  congruence,  intensity  and 
denial  to  various  scale  content.  RT  also  reflected  the  experimental  conditions  and 
Indicated  scale  difficulty  and  validity,  but  was  not  found  to  be  related  to  reaction 
time  speed  or  reading  measures. 


